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1. Executive Sunmary 


1.1 Introduction 

The objective of the Performance Measurement (PM) Module is to 
provide a computer-based interactive system for collecting and 
analyzing data from a wide variety of experiments. 

The challenge in the design of the PM Module is to provide a 
system that will be broad and flexible enough to handle a wide variety 
of experiments and performance measures, some of which have not yet 
been conceived. 

This document describes the design of the Performance 
Measurement Module. It focuses primarily on what the PM Module would 
do, and what it would look like to the user. The PM Module as 
described here could take several man-years to develop. This report 
suggests an evolutionary approach to the implementation of the PM 
Module. With such an approach an operational baseline PM Module could 
be running within a few months. 


1.2 Overview of the PM Module: DAT/STAT 

The Performance Measurement Module consists of two subsystems: 
a Data Analysis and Transformation (DAT) subsystem and a Statistical 
Analysis (STAT) subsystem. These subsystems are linked by a 
Consistent File System. See Figure 1.1. Files containing raw 
experimental data, such as time histories, are read and processed 
using the DAT subsystem, producing files of performance measures. 
These files, in turn, would be read and processed by the STAT 
subsystem. 

The DAT subsystem contains printing and plotting commands for 
examining raw data and derived performance measures. The STAT 
subsystem contains printing and plotting commands for examining the 
performance measures and the statistical summaries. 

There are primarily two reasons for dividing the PM Module 
into these two subsystems. First, the primary functions of these two 
subsystems are quite distinct: data manipulation versus statistical 
testing, although their secondary functions may indeed overlap. 
Second, it appears likely that it will be possible to purchase a 
statistical analysis package which would serve quite well as the STAT 
subsystem. 

Overviews of the DAT and STAT subsystems are presented below 
in Sections 1.2.1 and 1.2.2., and an overview of the Consistent File 
System is presented in Section 1.2.3. A plan for the evolutionary 
development of the PM Module is presented in Section 1.3. The DAT 
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DAT: DATA ANALYSIS & TRANSFORMATION 

STAT : statistical ANALYSIS 

Figure 1.1 Functional Block Diagram of the PM Module 
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subsystem and programming language are described in Sections 2 and 3, 
respectively. The STAT subsystem is described in Section A, and the 
Consistent File System for the PM Module is described in Section 5. 


1.2.1 Overview of the DAT Subsystem 

The purpose of the Data Analysis and Transformation (DAT) 
subsystem is to provide a tool for the scientific investigator to 
examine the data from his experiments. A useful analogy for this 
subsystem is a programmable matrix calculator with a graphic display. 
The current form of this subsystem has evolved over the past several 
months, and is now based strongly on BBN's RS/1 data management 
system. The subsystem began as a unified collection of relatively 
fixed analysis programs, and has evolved to the current more 
integrated and flexible form. 


The DAT subsystem, which is operated using a simple 
English-like command language, assists the user in managing 
two-dimensional data tables, in visualizing this data through graphs, 
and in preparing the data for statistical analysis via the Statistical 
Analysis (STAT) subsystem. In addition, the DAT subsystem includes a 
programming language which permits the creation of new procedures. 
There are two major purposes for these user-defined procedures. 
First, they enable the user to automate a sequence of analysis steps, 
such as reading a file, computing some measures, and plotting the 
results. Second, using the DAT programming language, the user can 
conveniently create and modify new performance measures and try them 
out on actual data. 

The functional description of the DAT subsystem is divided 
into two parts; a description of the DAT commands and a description 
of the DAT programming language. 


1.2.2 Overview of the STAT Subsystem 

The purpose of the Statistical Analysis (STAT) subsystem is to 
provide a tool for the scientific investigator to perform statistical 
tests and make statistical inferences from his data. 


As part of the design of the PM Module, a survey was made of 
existing, commercially available statistical analysis packages. It 
was hoped that such a ^ package could be incorporated into the PM 
Module, thereby avoiding needless extensive software development. 

In order to be considered for use as the STAT subsystem, a 
statistical package would have to meet the following four basic 
requirements; 
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(11 includes s letge range of standard statistical 
* ^analyses. Including analysis of variance. 

(2) Runs interactively. 

(3) Runs on a CDC Cyber-175 computer. 

( 4 ) Provides for data transformation. 


The fourth -quirement is plc"lge“ could 

^ri"f!ir”;n°tSal!r all IL -l-jl-enf, 

!r‘4n"%S?!ona;y^Vnrel with the fj-t ^tep being - Jasellne^PM 
i"ndtif dfti ?fanrfo%at!on"ra;aiilities of the SThT subsystem 
suffice . 

currently, , there are two candidate statistical analysis 
packages under consideration: 

(1) P-STM 78 produced by P-STAT, Inc. of Princeton, 

New Jersey. 

(51 SIPS produced by the Department of Statistics of 
‘ ’ Sregon i?Ste University, Corvallis, Oregon. 

A number of other Packages were If ' th if S^nm^n?? 

eliminated for one reason or another. I ® ‘ ^ In „ote detail, we 

we present the requireinents of the stat su y recommendation, 

compare the various candidate pMkages, ana m ^nft to the 

LR§ sta?f anr;!n"brbrser?rpa?t"nn experience with the pacKages 
gninid “'a ?rlal period which is just now beginning. 

123 Overview of the Consistent File System 

The. DAT and STAT aubsystems are linked^by_^a Consistent Pile 

System. This file system and STAT subsystems. The 

output of the --Pf system is to maintain the 

primary task of the Consistent i!i of a 

correspondence between th identities of the data (e.g. 

particular performance measure) and the identities 

the name of the performance measure). 

_ 4 T?i 1 p SvsteiTi fot thG PM Module has 

The design of tergal file structures and the 

two distinct parts: the Internal file structures means 

design of external file struct * . g g^c ; how does one identify 

1lnd°%°%ar?!lula?"da?™ «l?hin a file. External file structures 
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means the choice of file naming and accessing conventions: how does 

one identify and retrieve a particular file for analysis. 

The Consistent File System, described in Section 5, includes 
internal file structures which are designed meet the following 
objectives: 

(1) Adaptable, so that a wide variety of experiments and analyses 

may be accommodated. 

(2) Self-documenting, so that the files may be shared by users 

with little contact with each other. 

(3) Expandable, so that a subset of the users can make an 

addition to the file system. 

(4) Upward con^atlble, so that features which are not 

incorporated during the initial development can be added 

later . 

(5) Backward compatible, so that existing file handling software 

(e.g. SIFT) can be used. 

(6) Convenient, so that reading and writing the files requires 

just a few lines of code. 

(7) Efficient, so that the computing resources are not unduly 

burdened . 

The Consistent File System also includes external file 
structures which simplify the task of identifying and retrieving a 
particular file for analysis. The system is ,in fact, a file-naming 
convention, supported by appropriate software, with multi-field file 
names. The fields would serve to identify the following attributes of 
the file: 

(1) The user who "owns" the file: his group and his name. 

(2) The experiment to which the file pertains: the name of the 

experiment, the subject and run number. 

(3) The type of file: time-series data, processed data, etc. 

(4) The version number of the file. 

These file naming conventions would probably also be very useful for 
keeping track of programs pertaining to the PM Module, such as FORTRAN 
source files, documentation files, etc. 
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1.3 Evolutionary Development of the PM Module 

The oreceding section (1.2) was a functional description of a 
1 .U PM Module Such a module could take several man-years 

^ This section describes an evolutionary approach to the 
t»olfm^nta?Ion o? the ^ Module. In this approach, the ultimate 

atsitr wluia remain as described above. The to this goal^ 

° .^ 11 1 /^ Ko iria an evolutionary development f beginning with a 

baseline PM^^M^ule of somewhat limited power, and preceding via a 

series of upgrades to a complete PM Module. 

There are two very important advantages to this evolutionary 

There are two vety ^ possible to have an operational 

guide the design of the later stages. 

In the next two sections, an overview of the baseline PM 
Module is given, followed by a tentative series of upgrades. 


1.3.1 Overview of the Baseline PM Module 

The purpose of the Baseline PM Module 
working system with a minimum development 
Baseline PM Module is shown in Figure 1.2 
statistical analysis package selected for 
some additional code so that the consistent 
utilized to read time history data files 
measurement files. 

The DAT subsystem implemented in this Baseline PM Module would 
w Tt would include a workable subset of the 
functions the DAT subsystem. It would be implemented by adding a 
leS neS functions (such as' FPT and plot functions) to the data 
transformation capabilities of the STAT subsystem. 


is to provide a useful, 
effort. The form of the 
, It consists of the 
the STAT subsystem, with 
file system could be 
and to write performance 


1.3.2 Evolutionary Upgrades to the Baseline PM Module 

The Baseline PM Module, as described in the previous section 
falls short of the full PM Module in four major areas: 

(1) interactive graphics, 

(2) interactive data transformation, 

(3) automated data transformation, and 

(4) improved conversational front-end. 
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Figure 1.2 Form of the Baseline Module 




once the Baseline PM Module is running, these three areas could be 
Upgraded to the level of the full PM Module* 

The first area to be upgraded would almost certainly be 
interactive graphics. It is widely recognized that graphical 
presentation of numerical results can be of tremendous value to t^ 
person exploring a set of experimental results. The Baseline PM 
Module provides a very limited capability in this area, it would be an 
appropriate place to begin the upgrade. 

The next area to be upgraded might be the data transformation 
capabilities. Whether the work began with interactive data 

??ansformation (i.e. building towards the DAT^ subsystem ^ 
language), or automated data transformation (i.e. building towards the 
DAT programming language), or both together, would depend on e 
experience gained with the early utilization of the Baseline PM 

module. 

The decision to add a more conversational front-end to the PM 
Module would depend on the type of people who were expected to V®® 
svstem. Such a front-end could be very attractive to the scientist 
who is relatively unfamiliar with computer-based systems. On the 
other hand, other users might find that it introduced an unnecessary 

overhead. 
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2. The DAT Subsystem 


2.1 DAT Subsystem Concepts 

The DAT subsystem provides a convenient and efficient way to 
collect, store, analyze and examine research data. The user provides 
the system with numeric or textual data in tabular form from either an 
existing file, or from the computer terminal keyboard. Using a simple 
command language, the user can revise the data tables, create new data 
tables, display the data tables, create graphs of the data, revise the 
graphs, display the graphs on the terminal, and save the tables and 
graphs as files for later use. 

Although the DAT subsystem's command language provides a 
powerful set of tools for data manipulation and analysis, a user may 
create new procedures using the DAT programming language. This simple 
programming language allows the user to perform ®11 the DAT 
subsystem's commands under program control. The two major purposes 
for these user-defined procedures is (1) to automate frequently 
performed sequences of analysis steps, and (2) to conveniently create 
and modify new performance measures. 

The DAT command language is described in this section (2.) , 
and the DAT programming language is described in the next section 
(3.). 

Although the DAT subsystem has little or no statistical 
analysis capabilities built into it, it is easy for the user to write 
data tables as files which would be read by the STAT subsystem. 


2.1.1 A Very Short Session 

The DAT subsystem is a powerful system which includes table 
structures for holding data, making graphs, etc. However, in this 
introductory session, the DAT subsystem will be used as a simple desk 
calculator. This session will introduce a number of concepts which 
will be explained later. 

When the DAT subsystem is ready to accept a command, it types 
the prompt character "#". From that point on, until the user types 
the GO character (ALT MODE or ESCAPE) , the system is waiting for the 
user to finish typing. When the GO character is typed, it begins to 
execute the command. It is not ready to accept another command until 
"#" is typed again. 

The following session uses only the TYPE and SET commands: 

#TYPE 3.14159 <G0> 

3.14159 
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♦TYPE PI <G0> 

EMPTY 

♦SET PI’ TO 3.14159 <GO> 

♦TYPE PI <GO> 

3.14159 

♦TYPE PI* 10 <GO> 

31.4159 

♦TYPE 'DIAMETER OF CIRCLE IS: ',10 <GO> 
DIAMETER OF CIRCLE IS: 

10 

♦SET RADIUS TO 5.7 <GO> 

♦TYPE NOCR 'RADIUS OF CIRCLE IS:', RADIUS,' 
diameter OF CIRCLE IS:', 2*RADIUS,' 

AREA OF CIRCLE IS:', PI*RADIUS**2 <GO> 
RADIUS OF CIRCLE IS: 5.7 
DIAMETER OF CIRCLE IS: 11.4 
AREA OF CIRCLE IS: 102.07 
♦QUIT <GO> 


2 . i> 2' CbiDBoni DAT Commands : TYPE, and' SETC’ 

The DAT subsystem manipulates and displays data by executing 
commands which the user types in to it. A common, command, which is 
used< in- the above very short session-, is the TYPE command,^ which, tells, 
the DAT' s'libsystem' to type a; value. The- user can. specify- several 
values to TYPE by separating the values by commas. Each value will be 
typed on a new line, unless the TYPE NOCR command is used. "NOCR" 

stands for no carriage return. The general form for the TYPE 

statement is: 

TYPE [NOCR] {value} 

where [...]■ indicates an optional argument, and {...} indicates that 
several arguments may be present separated by commas. 

The SET command is used to change the value of a variable; it 

does not cause anything to be typed out. The general form of the SET 

command is: 


SET variable TO expression 

A useful, although not necessary, feature of the DAT subsystem would 
be to allow the use of "=" interchangeably with the set command : 

variable = expression 

Variables are used to store results that may be used later. 
The user need not do anything special to make a variable exist; the 
DAT subsystem takes care of that when a variable is used for the ^ first 
time. It also has an explicit representation for data which is not 
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known, the value EMPTY. This is the value all variables have when 
they are first created. 


2.1.3 Values 

There are four different kinds of data values in the DAT 
subsystem: 

FIXED - An integer, e.g. 4, -5, 0, 15123 

FLOAT - A number with a decimal point, or written with 
an "E" followed by a number, to mean "times 10 to 
the number." For example, 3.14159, 3.5E12, 2E-7 

TEXT - Text values are strings of typed characters, 
enclosed in single quotes. For example, ''THIS IS 
A TEXT.', ' 3 ', 'RADIUS OF CIRCLE IS:' 

EMPTY - This is a special value. It is used to 

indicate that the value is not known. EMPTY may 
be used in computations, but it typically produces 
an EMPTY result. For example, EMPTY + 1 is EMPTY. 


2.1.4 Expressions: Operators and Functions 

Values may be computed in the DAT subsystem by means of 
expressions. An expression is formed by using operators and functions 
to combine constants and variables. 

The order of evaluation of an arithmetic expression may be 
explicitly indicated by parentheses. Where this is not done, 
expressions are evaluated in the following order: 

(1) ** (exponentiation) and - (unary minus) 

(2) * and / (multiplication and division) 

(3) + and - (addition and subtraction) 

Within these constraints, expressions are evaluated from left to 
right. Thus A/B*C is evaluated as (A/B)*C. The only exception to 
this left-to-r ight rule is exponentiation where A**B**C is evaluated 
as A** (B**C) . 

There are four categories of operators available in the DAT 
subsystem: 

Arithmetic: +, -, *, /, ** 
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Relational: EQ/ NE, LT, GT, LE, GE 

Logical: AND, OR, NOT 

Text: CAT (concatenation) 

There are six categories of functions available in the DAT 
subsystem: 

Trigonometric; SIN, COS, TAN, ... 

Numerical: SQRT, EXP, LOG, MAX, MIN, ... 

Text: CHARS, CAT, ... 

Data Type; TNUM, TYPE, EMPTY, NUM.TO.TEXT, ... 

Graphic: DLINE, WIOVE, ERASE, ... 

Miscellaneous; GETFILE, PUTFILE, YESANSWER, ... 


2.2 DAT Tables 

One of the most important entities in the DAT subsystem is the 
data table. Figure 2.1 is a typical table. It contains data from a 
realistic (though fanciful) data set concerning the influence of a 
drug called TAIL-GRO on the growth of armadillo tails. 


2.2.1 Making a Table via the Keyboard 

This table was constructed using the MAKE TABLE command. This 
command prompts the user for the title of the table, row names and 
column headings, and for the data values comprising the body of the 

table. 

The MAKE TABLE dialogue that led to the table in Figure 2.1 is 
given below; 

#MAKE TABLE ARM A <G0> 

TITLE OF TABLE? ARMADILLO TAIL-GRO DATA <GO> 

ARE THERE ROW NAMES? YES <GO> 

ARE THERE COLUMN HEADINGS? YES <G0> 

ENTER COLUMNWISE? YES <GO> 

NAME OF ROW 1 = A352C <G0> 

NAME OF ROW 2 = A601D <G0> 

NAME OF ROW 3 = A705R <G0> 

NAME OF ROW 4 = A852R <GO> 

NAME OF ROW 5 = A965D <G0> 


12 



ARMA 3C X 7R ARMADILLO TAIL-GRO DATA 


1. A352C 1 

1 PERSONALITY 
1 FRIENDLY 

2 DOSE (MG) 
8.2 

3 TAIL (CM) 
7.9 

2. A601D 1 

1 QUIET 

13.2 

24.3 

3. A705R 1 

1 NASTY 

25.3 

26.3 

4. A852R 1 

1 QUIET 

1 7.9 1 

8.2 

5. A965D 1 

1 NASTY 

1 36.1 1 

35.5 

6. A572C 1 

1 FRIENDLY 

I 9-6 1 

7.7 

7. A631D 1 

1 FRIENDLY 

I 11.5 1 

27.5 


Figure 2.1 A Typical Table 


name of row 6 

NAME OF ROW 7 
NAME OF ROW 8 
COL 1 HEADING 
COL 1 ROW 1 = 
COL 1 ROW 2 = 
COL 1 ROW 3 = 
COL' 1 ROW 4 = 
COL 1 ROW 5 = 
COL 1 ROW 6 = 
COL 1 ROW 7 = 
COL 2 HEADING 
COL 2 ROW 1 = 
COL 2 ROW 2 = 
COL 2 ROW 3 = 
COL 2 ROW 4 = 
COL 2 ROW 5 = 
COL 2 ROW 6 = 
COL 2 ROW 7 = 
COL 3 HEADING 
COL 3 ROW 1 = 
COL 3 ROW 2 = 
COL 3 ROW 3 = 
COL 3 ROW 4 = 
COL 3 ROW 5 = 
COL 3 ROW 6 = 
COL 3 ROW 7 = 
COL 4 HEADING 


= A572C <GO> 

= A631D <GO> 

= next <go> 

= personality <go> 

FRIENDLY <GO> 

quiet <G0> 

NASTY <GO> 
quiet <G0> 

NASTY <GO> 
friendly <G0> 
friendly <G0> 

= DOSE (MG) <GO> 

8.2 <GO> 

13.2 <GO> 

25.3 <GO> 

7.9 <GO> 

36.1 <GO> 

9.6 <GO> 

11.5 <GO> 

= TAIL (CM) <GO> 
7.9 <GO> 

24.3 <GO> 

26.3 <GO> 

8.2 <GO> 

35.5 <GO> 

7.7 <GO> 

27.5 <GO> 

= EXIT <GO> 


2.2.2 Making a Table from a File 

In the PM Module, it is particularly Sle'^^^'^For 

be able make% table £ro» an exjst ng £ile.^^ For 

ta?a Flies ana cb^nands 

for reading and writing these files. 

The general form of the command to open a file is; 

OPEN FILE filename 


The general form of the command to close a file is: 

CLOSE FILE filename 

The general form of the command to make a table from a file is 

make table tablename FROM filename 


14 



The general form of the command to make a file from a table is: 

MAKE FILE filename PROM tablename 

Each of these commands invokes a dialogue specifying the portions of 
the table or file to read and write. 


2.2.3 Changing Data in a Table 


The DAT subsystem provides a simple 
changes to the data in a table. In order to 
armadillo A631D in table ARMA (Figure 2.1), 
CHANGE command as follows: 


mechanism for making 
change the dose for 
the user would use the 


#CHANGE ARMA <G0> 
ROW? 7 <G0> 

COL? 2 <G0> 

VALUE? 17.5 <G0> 
ROW? EXIT <G0> 


The resulting table would then be as shown in Figure 2.2. Notice that 

last row of column 2 has been changed from 11.5 to 

■i- / • ^ • 


2.2.4 Displaying a Table 

4 .K 1 - command is used to display a table. The command 

that would display the table ARMA as shown in Figures 2.1 and 2.2 is: 

♦DISPLAY ARMA <G0> 


2.2.5 Adding and Deleting Data in an Existing Table 


The ADD COLS and ADD ROWS commands are used to add data to an 
* Executing one of these commands re-enters the MAKE 
^ example, adding a column of scale shapes to the 
table ARMA is performed via the following dialogue: 


♦ADD COLS TO ARMA <G0> 

COL 4 HEADING = SCALE 
SHAPE <G0> 

COL 4 ROW 1 = TRIANGLE <G0> 

COL 4 ROW 2 = TRIANGLE <GO> 

COL 4 ROW 3 = TRIANGLE <G0> 

COL 4 ROW 4 = TRIANGLE <G0> 

COL 4 ROW 5 = SQUARE <G0> 

COL 4 ROW 6 = DIAMOND <G0> 

COL 4 ROW 7 = DIAMOND <GO> 
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ARMA 3C X 

7R 

ARMADILLO TAIL- 

-GRO DATA 


1 


1 PERSONALITY 

1 

2 DOSE 

(MG) 1 3 TAIL (CM) j 

1 



1 



1 1. A352C 


j friendly 

L 

8.2 

1 7.9 1 

\ 2. A601D 


QUIET 

1 

13.2 

1 24.3 1 

1 3. A705R 

"i 

1 NASTY 

i_ 

25.3 

1 26.3 1 

1 4, A852R 

*T 

1 QUIET 

1 

7.9 

1 8.2 1 

1 5. A965D 


1 NASTY 

1 

36.1 

1 35.5 1 

1 6. A572C 


1 friendly 

’i' 

9.6 

1 7.7 1 

1 7. A631D 

"I 

1 friendly 

T 

17.5 

1 27.5 1 


Figure 2.2 A Typical Table Revised 
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COL 5 HEADING = EXIT <GO> 


This new column of data could be displayed using the table 
portion form of the DISPLAY command: 

#DISPLAY COL 4 OF ARMA <G0> 

and the table portion would be displayed as shown in Figure 2.3. 

Similarly, data on an additional set of 15 armadillos would be 
added via the ADD ROWS command. These new rows of data could be 
displayed via the command: 

#DI SPLAY ROWS 8 TO 22 OF ARMA <G0> 

and the table portion would be displayed as shown in Figure 2.4. 

Data may be deleted from an existing table via the DELETE 
command. It turns out that rows 6 and 17 of table ARMA are identical. 
Row 17 may be deleted using the following command: 

#DELETE ROW 17 OF ARMA <GO> 


2.3 Advanced Table Concepts 

Some advanced table concepts are presented in this section, 
the most important of which is the notion of table portions. An 
alternate notation for tables is also presented, along with 
information on renaming and deleting tables. 


2.3.1 Table Portions 

One of the most powerful concepts that is available to the DAT 
subsystem user is the table portion. A table portion is a rectangular 
subset of the cells of a table. It may be described by specifying 
certain rows, certain columns, or certain cells of a table. Here are 
some examples of table portions used to display only part of a table 
of data: 

DISPLAY ROWS 1, 3 TO 5, 8 OF DATATAB <G0> 

DISPLAY COLS 5 TO 9, 1 OF DATATAB <G0> 

DISPLAY ROWS 1, 2 OP COLS 6, 9 OF DATATAB <GO> 

DISPLAY COL 2 OF ROWS 1 TO 3 OF DATATAB <G0> 

The statements shown above specify a portion of the table named 
DATATAB by listing the rows, columns, or both which are to be 
included. The four ways to specify such a table portion are: 
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Figure 2.3 A New Column of Data 
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ARMA 4C X 22R ARMADILLO TAIL-GRO DATA 


1 PERSONALITY 2 DOSE (MG) 3 TAIL (CM) 4 SCALE 

SHAPE 


8. A536R 

11 

SICK 

1 24.7 

1 31. 

1 SQUARE 

9. A867R 

11 


1 31.2 

1 33.8 

1 SQUARE 

10. A945C 

11 

SICK 

1 8.9 

1 8.9 

1 TRIANGLE 

11. A856R 

11 

QUIET 

1 9.3 

1 7.8 

1 DIAMOND 

12. A978D 

11 

NASTY 

1 29.1 

1 32.9 

1 DIAMOND 

13. A877R 

11 

DUMB 

1 14.8 

1 34.1 

1 DIAMOND 

14. A938C 

11 

QUIET 

1 36.2 

1 35.6 

1 DIAMOND 

15. A937R 

11 

FRIENDLY 

1 40.2 

1 36.9 

1 SQUARE 

16. A749C 

7i 

NASTY 

I 35.2 

1 35.2 

1 SQUARE 

17. A572C 

'ir 

FRIENDLY 

1 9.6 

1 7.7 

1 DIAMOND 

18. A462C 

m 

SICK 

1 7.8 

1 8 . 

1 TRIANGLE 

19. A465R 

11 

QUIET 

1 9.2 

1 8.3 

1 DIAMOND 

20. A465D 

11 

DUMB 

1 17.9 

1 30.9 

1 TRIANGLE 

21. A114R 

11 

FRIENDLY 

1 46.1 

1 38.7 

1 SQUARE 

22. A593D 

11 

NASTY 

1 34.1 

1 34.8 

1 SQUARE 


Figure 2.4 New Rows of Data 
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(1) ROW[S] <list> OP tablename 

t7.\ COLfSl <list> OF tablename 

(3 ROW S <list> OF COL IS] <list> OF tablename 

( 4 ) COL[S] <list> OF ROW[S] <list> OF tablename 

The "<list>" construct contains numeric values or entries of the form; 

<numeric-value> TO <numer ic-value> 

Entries in the list are separated by commas. 

In addition to extracting a table portion by 

portion is; 

tablename WHERE <expression> 

An example of the use of this feature is included in the following 
section. 

2.3.2 Extracting Table Portions 

The MAKE TABLE coiranand may be used in '=?n3“"=‘jo|' 
portion to extract a table portion from^an existing table^and_^maXe^a 

aJLduio *da?a 'tn gust the oases^nvolving high doses, following 

commands could be used to make a new table and display it. 

#MAKE TABLE HIGHDOSE FROM ARMA WHERE COL 2 GT 10 <GO> 

#DISPLAY HIGHDOSE <GO> 

The resulting table HIGHDOSE is shown in Figure 2.5. 


2.3.3 Setting Table Portions 

The SET command can be used to change the value of a variable 
(see Section 2.1.2), or a single cell in a table, or all the cells 
a table portion. For example the commands; 

#SET ROW 3 COL 2 OF DATATAB TO 2.71828 

#SET ROW 2 COL 3 OF DATATAB TO ROW 3 COL 2 OF DATATAB 

would set row 3 column 2 and then row 2 column 3 of table DATATAB to 
2.71828. Similarly, the commands; 

#SET COL 5 OF DATATAB TO 1.414 <GO> 
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HIGHDOSE 4C X 14R 


1 

1 1. A601D 1 

1 PERSONALITY j 2 DOSE 
1 QUIET 1 13.2 

(MG) 

3 TAIL (CM) 
1 24.3 

4 SCALE 
SHAPE 

TRIANGLE 1 

1 2. A705R 1 

1 NASTY 

1 25.3 


1 26.3 

TRIANGLE 1 

1 3. A965D 1 

1 NASTY 

1 36.1 


35.5 

SQUARE 1 

1 4. A631D 1 

1 FRIENDLY 

1 17.5 


27.5 

DIAMOND 1 

1 5. A536R 1 

1 SICK 

1 24.7 


31. 

SQUARE 1 

1 6. A867R 1 


1 31.2 


1 33.8 

SQUARE 1 

1 7. A978D 1 

1 NASTY 

1 29.1 


1 32.9 

DIAMOND 1 

1 8. A877R 1 

1 DUMB 

1 14.8 


1 34.1 

DIAMOND 1 

1 9. A938C 1 

1 QUIET 

1 36.2 


1 35.6 

DIAMOND 1 

|10. A937R 1 

1 FRIENDLY 

1 40.2 


1 36.9 

SQUARE 1 

111. A749C 1 

1 NASTY 

1 35.2 


1 35.2 

SQUARE 1 

112. A465D 1 

1 DUMB 

1 17.9 


1 30.9 

TRIANGLE 1 

113. A114R 1 

1 FRIENDLY 

1 46.1 


1 38.7 

SQUARE 1 

Il4. A593D 1 

1 NASTY 

1 34.1 


1 34.8 

SQUARE 1 


Figure 2.5 

Extracting 

a Table Portion 
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would set all the 


cells in column 5 of table DATATAB to 1.414. 


2.3.4 Sorting a Table 

rpKia nAT subsystem has a special command for sorting a table by 
.K in a specified row or column. The SORT command sorts in 

the ^sta in ty.- user types DESCENDING at the end of the 

ascending order following command would sort the data in 

?aS?rHiGHDOsl br^Se^ise'^ount and lisplay the result: 


♦SOFT HIGHDOSE BY COL 2 <G0> 

♦DISPLAY HIGHDOSE <GO> 

The resulting table is shown in Figure 2.6. 


2.3.5 An Alternative Notation for Tables 


The DAT subsystem ^®®°^2irR0w"l^C0L^2^0F TBL .^^Thus^ourmly 
think^of a^tabim\^tSo-dirnensional array, and use this alternative 
notation to address the data items. 


2.3.6 Directory of Tables 

Th. names and sizes of all the tables that are created are 
held in f Special table called the DIRECTORY. To get a complete 
of the directory, the user can type: 


♦DIS DIRECTORY <G0> 

A typical directory is shown in Figure 2.7. 


2.3.7 Renaming and Deleting Tables 


T 4 . a aood idea to keep the DIRECTORY "cleaned-up" , by 
giving tables short'but meaningful names, and by deleting unwanted 
?abiel. TO rename a table, the RENAME command is used: 


♦RENAME TABLE FOO TO ^STATS^ <G0> 

TO delete a table, the DELETE command is used 


♦delete table < table name> <G0> 


This will remove the table 
corresponding file on the disk, 
is lost. 


from the directory, and 
Once the table is deleted. 


delete the 
the data 
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HIGHDOSE 4C X 14R 


1 1. A601D 1 

1 PERSONALITY | 2 DOSE (MG) [ 3 TAIL (CM) | 4 SCALE 

1 1 I SHAPE 

1 QUIET 1 13.2 1 24.3 | TRIANGLE 1 

1 2. A877R 1 

1 DUMB 

1 14.8 

1 34.1 

1 DIAMOND 

1 3. A631D 1 

1 FRIENDLY 

1 17.5 

1 27.5 

1 DIAMOND 

1 4. A465D 1 

1 DUMB 

1 17 . 9 

1 30.9 

1 TRIANGLE 

1 5. A536R 1 

1 SICK 

1 24.7 

1 31. 

1 SQUARE 

1 6. A705R 1 

1 NASTY 

1 25.3 

1 26.3 

1 TRIANGLE 

1 7. A978D 1 

1 NASTY 

1 29.1 

1 32.9 

1 DIAMOND 

1 8. A867R I 

1 

1 31.2 

1 33.8 

1 SQUARE 

1 9. A593D 1 

1 NASTY 

1 34.1 

1 34.8 

1 SQUARE 

|10. A749C 1 

1 NASTY 

1 35.2 

1 35.2 

1 SQUARE 

111. A965D 1 

1 NASTY 

1 36.1 

1 35.5 

1 SQUARE 

Il2. A938C 1 

1 QUIET 

1 36.2 

1 35.6 

1 DIAMOND 

|13. A937R 1 

1 FRIENDLY 

1 40.2 

1 36.9 

1 SQUARE 

|14. A114R 1 

1 FRIENDLY 

1 46.1 

1 38.7 

1 SQUARE 


Figure 2.6 Sorting a Table 



2C X 15R 


1 TABLE 


1 

5 

11 

ARMA 

1 2 


r 

6 

ir 

AXES OF ARMAPLOT 

1 2 


r 

7 

11 

CURVES OF ARMAPLOT 

1 ^ 


r 

8 

ir 

DATA OF ARMAPLOT 

1 2 


r 

9 

71' 

DATATAB 

1 44 


r 

10 

71' 

AXES OF HI PLOT 

1 2 


r 

11 

7r 

CURVES OF HI PLOT 

1 2 


r 

12 

7i 

DATA OF HI PLOT 

1 2 


r 

13 

77 

TBL 

1 3 


r 

14 

’II 

HIGHDOSE 

1 2 


r 

15 

11 

STATS 

1 ^ 



Figure 2.7 A Directory 
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2.4 Graphs 


The Dat subsystem provides a powerful yet convenient mechanism 
for creating two-dimensional graphs from data which is stored in 
tables. 


2.4.1 Exai^le of a Graph 

Figure 2.8 is an example of a simple graph. The data for this 
graph was extracted from the data in columns 2 and 3 of table ARMA, 
which is displayed in Figure 2.2. 

2.4.2 Making a Graph from a Table 

This graph was constructed using the MAKE GRAPH command. This 
command prompts the user for the title of the table, the x and y axis 
labels, the x and y axis forms (linear or log) , the sources of the x 
and y values, the curve symbols, and curve labels. 

The MAKE GRAPH dialogue that led to the graph in Figure 2.8 is 
given below; 


#MAKE GRAPH ARMAPLOT <G0> 

TITLE OF GRAPH? EFFECT OF TAIL-GRO ON TAIL LENGTH <G0> 

X AXIS LABEL? DOSE (MG) <G0> 

X AXIS LINEAR (LIN) OR LOG? LIN <G0> 

Y AXIS LABEL? TAIL 
LENGTH <G0> 

y AXIS LINEAR (LIN) OR LOG? LIN <G0> 

X VALUES FRCM TABLE (T) OR ENTER (E)? T <G0> 

ENTER TABLE PORTION; COL 2 OF ARMA <G0> 

ENTER SOURCE OF CURVE 1; TABLE (T) OR FUNCTION (F)? T <G0> 

ENTER TABLE PORTION; COL 3 OF ARMA <G0> 

CURVE SYMBOL? SQUARE <G0> 

CURVE LABEL? ARMADILLO DATA OF 1/6/79 <G0> 

CONNECTED -- YES(Y) OR NO(N)? N <GO> 

EXIT(E), NEW X-VALUES(X) OR Y-VALUES (Y) FOR NEXT CURVE; E <G0> 


2.4.3 Displaying a Graph 

The DISPLAY command is used to display a graph. The command 
that would display the graph ARMAPLOT as shown in Figure 2.8 is; 


♦DISPLAY ARMAPLOT <GO> 




2.4.4 Editing a Graph 

The DAT subsystem does a reasonable job in selecting the 
proper defaults for upper and lower limits on axes etc. However, 
there are a number of parameters which may be changed through the 
graph editing commands. There are two sets of parameters which can be 
changed by the editing commands; axis information and curve 
information. There are five variables which may be set for each axis. 
The table below shows the name of the variables and their assigned 
values: 

VARIABLE 

LABEL 
LINLOG 
LOW 
HIGH 
UNITS 

These variables may 
variable. 

There are five variables which pertain to the curves; 


VALUE 


USE 


<text> name of axis 

LIN or LOG linear or log axis 

<numeric value> lowest value shown 
<numeric value> highest value shown 
<numeric value> units per tick 

be TYPEd or SET like any other DAT subsystem 




VARIABLE VALUE 


USE 


LABEL 

SYMBOL 

CONNECTED 

LINE 

FUNCTION 


<text> in legend 

<letter or plotting symbol 

plotting symbol > 

YES or NO connected by lines? 

<text> type of connecting line 

(see below) 

<text> 


The LINE specification can be one of the following; 


SINGLE 

DOUBLE 

DOT 

DASH 

DOT DASH 
DOT DOT DASH 

FUNCTION accepts an expression in X from which it generates a set of Y 
values, e.g. 


"X**2 + SIN(X/2)" 

The curve variables may be accessed using the following syntax; 
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SET <variable> OP CORVE <number> OP <graph name> TO <value> 

When a graph is first constructed, these variables are all given 
values. 

in the following dialogue, the user adjusts the LOW and HIGH 
values of ARMAPLOT and then DISPLAYS the revised graph: 

#SET LOW OF Y AXIS OF ARMAPLOT TO 0 <GO> 

JsBT HIGH OF Y AXIS OF ARMAPLOT TO 40 <GO> 

#DIS ARMAPLOT <GO> 

The revised graph is shown in Figure 2.9. 

2.4.5 Adding and Deleting Curves 

nnrp a araoh has been constructed, curves may be added or 
deleted. L?efinfc«?es from a graph is similar to deleting rows or 
cllui^rfrom a table. The user simply types: 

♦delete CURVE [S] <list> OF <graph name> <GO> 

The list or curves is a list of curve ‘^they'^are^listed in the 

deleted. The curves are numbered in the order y 
legend below the graph, starting with 1. 

Adding curves to a graph is similar to making the graph in the 
first place. The user simply types: 

♦ADD CURVE [S] TO <graph name> <GO> 

This will re-enter the make graph dialogue at the point where the X 
specifications are determined for a curve. 

Anv curve additions or deletions have a permanent effect. The 
original grap^wfll have been altered. Subsequent displays of the 

graph will show the new form. 


2.4.6 Pitting Polynomials to Graphs 


The OAT -bsystem allows one to fit ^ jolynomial^to^a curve.of 

^cu^?e^tf %he" V f'^^ive^ "e ^1?? 

command may be used; to fit ® P 2.io is a graph called GROPLOT 

contaSling a'^curve of^data points to which a straight line has been 
fit via the following command: 
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5. 10. 15. 20. 25. 30. 35. 00. 05. 50. 

DOSE(HG) 

□ ARMADILLO DATA OF 1/6/79 


Figure 2.9 A Simple Graph Revised 





GROPLOl 



+ DELTA TAIL(CM) 

0.189049*X +0.812604 


Figure 2.10 A Polynomial Fit to a Graph 
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#FIT LINE TO CURVE 1 OF GROPLOT 

2.5 Using DAT Subsystem Procedures 

DAT subsystem procedures are written in the DAT programming 
language which is described in the next section. These procedures may 
be invoked either directly at the DAT command level or within another 
procedure . 

There are two methods of invoking a procedure: (1) the CALL 
statement/ e.g. 

#CALL GETLINE (STRING, CHAN) <G0> 

#CALL ERASE <G0> 

and (2) as a function call, e.g. 

#SET X TO SIN(Y) <G0> 

#X = SIN(Y) <G0> 

#TYPE COS(Y) <G0> 

The call statement is used if the procedure does not return a result. 
Such a procedure is run for its side-effect, such as changing the 
value of its arguments or performing input or output. 


2.5.1 Example of a Procedure 

SPIRO is a procedure that draws fancy spirals, such as shown 
in Figure 2.11. This figure was produced by calling SPIRO as follows: 

#CALL SPIRO(123, 123, 1.02, 1000) <GO> 

The procedure SPIRO is written in the DAT programming language; its 
code is shown in Figure 2.12. Detailed information on the DAT 
programming language is presented in the next section. 
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Figure 2.11 A Spiral Drawn by Procedure SPIRO 
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0 <TOP OP TEXT> 

1 /* SPIRO */ 

2 PROCEDURE (RAD, ANGL,ATT , NUM, XO, YO) ; 


3 

A 

/* THIS PROCEDURE DRAWS 

FANCY SPIRALS */ 

4 

5 

CALL ERASE; 

/* 

CLEAR THE SCREEN */ 

6 

ANG=0; 



/ 

8 

IF NARGS=4 

/* 

OPTIONAL ARGUMENTS 

9 

THEN DO; X0=512; Y0=390; 

END; 


10 



LOOP NUM TIMES */ 

11 

DO I = 1 TO NUM; 

/* 

12 

Xl=RAD*COSD (ANG) +X0; 



13 

Y1=RAD*SIND (ANG) +Y0 ; 



14 

IF Xl>1023 OR Yl>780 

OR XKO 

OR YKO 

15 

THEN DOEXIT; 

/* 

EXIT IP OFF SCREEN 

16 




17 

CALL DLINE(X0,Y0,X1,Y1) ; /* 

DRAW THE LINE */ 

18 




19 

X0=X1; 

/* 

UPDATE VARIABLES */ 

20 

Y0=Y1; 



21 

RAD=RAD*ATT; 



22 

ANG=MOD ( ANGL+ANG ,360) 

• 


23 

END; 



24 




25 

END; 



26 

<BOTTOM OF TEXT> 




Figure 2.12 Pr<x:edure SPIRO 
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3* The DAT Prograaaiing Language 


3.1 Introduction to the DAT Programming Language 

The DAT progtamnlng l»"9uage permlts^the^uset 
procedures. *”they° ewble the'^user to automate a sequence of 

I'n^sirsteps^i^ch al rLdinq a n 

SL°r^Sn‘''lo^^rni:^;i/«rat4 “a^r^od11y°ne«%e?forma„ce measures and 
try them out on actual data. 

euhsyster -^^0^ 

interchangeable; programming emoloved within procedures. In 

command level, and DAT commands ®^P^°^®°essible from either the 

addition, the data (tables an g ^ Whatever manipulation 

DAT °thf “SoLand le?el, can also be done automatically, 

within a procedure. 


3.2 Data Structures 

All of the DAT subsystem data types and data structures are 
accessible from the DAT programming language. 


3.2.1 constants. Variables and Expressions 

1- ra tvoes of data values available in the DAT 

subsystem, and are described in Section 2.1.3. 

may be assigned any of the r yp before they are referenced 

in a procedure should be assigned a ^lue before they 3^.^. ^ 

in a statement or expression. This process is^^^ 

variable. An uninitialize before they are used, however; 

variables do not have to be . iruled fL the first time, 

space is allocated to a variable when it is usea 


A variable is referred to by a variable name 
consist of an arbitrarily long string of characters, 
which must be alphabetic. 


This name may 
the first of 


IS 
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variables. The expressions, operators and functions available are the 
same as in the DAT subsystem, and are described in Section 2.1.4 


3.2.2 DAT Tables 

The data tables available in the DAT subsystem are also 
available in the DAT programming language. In the DAT programming 
language, however, tables are made either from a file, or from an 
existing table portion. The MAKE TABLE dialogue is not available. 
One could create a table by MAKEing a table, and assigning the row and 
column names, and data values via the SET statement. Similarly, the 
CHANGE TABLE command is not available; one could merely SET the data 
values to new values. In general, the DAT subsystem commands which 
produce a dialogue are not available; the rest are available. 


3.2.3 Graphs 

The graphs available in the DAT subsystem are also available 
in the DAT programming language. As with tables, however, the MAKE 
GRAPH dialogue is not available. One would create a graph by MAKEing 
a graph from a table, and assigning the title, axis labels, etc. via 
the SET command. Similarly, the dialogue following the DELETE CURVE 
and ADD CURVE commands are not available; one would merely SET the 
parameter values. In general, as with tables, the DAT subsystem 
commands which produce a dialogue are not available; the rest are 
available. 


3.3 Programming Statements 

The set of DAT programming statements is small and powerful. 
There are only six types of statements; 

assignment statements, 

input/output statements, 

conditional statements (IF-THEN-ELSE) , 

block and loop constructs (DO- END block, DO loop, DO-WHILE) , 
unconditional jumps (GO TO) . 
and procedure calls and returns. 

All DAT procedures (programs) are constructed from combinations of 
these six elements, plus any of the command-level The first five types 
of statements are covered in this section, and the last, procedure 
calls and returns are covered in the next section 3.4. commands. 

In general, DAT programming statements can be written in any 
format. Statements can be broken by carriage returns or by multiple 
spaces between words. At least one space is required between separate 
words, and each statement must be terminated with a special character 
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- a semicolon C;"). The semicolon is analogous to the <G0> character 
at the coiranand level. 


3.3.1 Assignment Statements 

Assignment statements 
SET statement is usedr or the 
of the SET statement is: 


may take one of two forms. Either the 
=" operator is used. The general form 


SET variable TO expression; 


An assignment statement using the 
variable = expression; 


operator takes the form: 


3.3.2 Input/Output Statements 


T ♦. anri rtiifout Statements constitute the means of getting 

if useflo? T?pl 

statement for output. 

The basic input statements in the DM programming language 

INPUT numeric-variable; 


arei 


INPUT TEXT text-variable; 

An input be^feSfby ^he ulef^ta 

the keyKard; and^ (2) it sets the variable named in the INPUT 
statement to the value entered. 

<GO> kev. The INPUT statement checks that tne vaxue x*> 
ofa FKED, and returns an error message if it is not. 

•. uif-h thp INPUT TEXT statement. Any text 

enrerea 

Tofl; ,ura^^tSrM Ir^r^Lfng^lafgfagraols'this Implicitly, 
there are two simple statements: 
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TYPE variable; 

TYPE NOCR variable; 

The first statement types out the value of any variable followed by a 
carriage return. The TYPE NOCR outputs the variable without the 
carriage return. 

The TYPE statement can output an arbitrarily long list of 
variables, if the variable names are separated by commas. A carriage 
return will be typed after each value. 


3.3.3 Conditional Stateaents 

A conditional statement is one that tests a condition and 
executes a command based on the results of the test. The basic 
conditional construct is the IF... THEN statement, and its adjunct, the 
ELSE statement. 

The IF... THEN statement can be formally expressed as: 

IF boolean-expression THEN statement; 

where "boolean-expression" is any expression (or single variable) 
which evaluates to TRUE or FALSE. Any legitimate expression 
containing relational or boolean operators meets this criterion, as 
does any function that yields a truth— value. 

The "statement" portion of the IF... THEN construction may be 
any legal DAT programming language statement that can be executed, 
including another IF... THEN statement, or a DO... END block (see 
3.3.4) . 

When the boolean-expression in an IF... ELSE statement is 
evaluated as FALSE, the THEN statement is not executed. When the 
boolean— express ion is TRUE, the THEN statement is executed. 

The ELSE statement is an extension of the IF... THEN statement 
that increases its power. It must immediately follow an IF... THEN 
statement. The general form of the IF .. .THEN .. .ELSE construction is: 

IF boolean-expression THEN statement; 

ELSE [statement]; 

where the statement following the ELSE is optional. ^ The effect of the 
ELSE statement is to specify an alternative action to be performed 
when the IF condition is FALSE. The statement following the ELSE is 
optional so that nested IFs and ELSEs may be paired even when some 
ELSE alternatives do not exist. 
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3.3.4 Block and Loop Constructs 

The DO... END block combines a group of statements into 
constructs that are treated logically as a single statement. Here is 
the general form of the DO... END block; 

DO; 

statement; 

statement; 


statement; 

END; 

where "statement" is any executable statement , including both simple 
and nested conditional statements. 

One of the most important uses of the DO... END block is to 

program a series of operations, based on the outcome of a conditional 
statement. For example; 

IF NOTEMPTY(X) 

THEN DO; 

TYPE X; 

SUMM=SUMM+X; 

END; 

ELSE TYPE 'EMPTY VALUE'; 

A DO... END block can also be used following an ELSE statement. 

The DO loop is a special form of the DO... END block which 

allows a group of statements to be executed in an iterative way. DO 
loops may be further classified into two distinct types; the basic DO 
loop and the DO WHILE loop. The basic DO loop repeats a set of 

operations a number of times, with the number of iterations specified 

in the DO statement. The DO WHILE loop is more open-ended; it 
continues its iterations an unspecified number of times, repeating the 
set of operations as long as the specified condition holds. 

The general form of the DO loop is as follows; 

DO counter = start-value TO quit-value [BY increment] ; 
statement; 
statement; 


statement; 

END; 


38 



"Counter" is a numeric variable that keeps count of the number of 
times the loop is repeated. "Start-value" and "quit-value" specify 
the number of iterations. The counter proceeds from the start-value 
to the quit-value by an increment of 1 each time the loop is executed, 
unless a different increment is specified in the "BY increment" 
option. 

The general form of the DO... WHILE loop is as follows: 

DO WHILE (boolean-expression); 
statement; 
statement; 


statement; 

END; 

where boolean expression is any expression that evaluates to TRUE or 
FALSE. The parentheses around the boolean-expression are required. 
The loop will continue to iterate until control returns to the first 
line and the boolean-expression is evaluated as FALSE. 

For greater flexibility, DO loops and DO... WHILE loops can be 
terminated before the normal quitting condition is reached by means of 
the DOEXIT and DONEXT statements. When a DOEXIT statement is 
encountered in a loop, control immediately passes out of the loop to 
the statement following the END. When a DONEXT statement is 
encountered in a loop, control jumps back to the first line of the 
loop and the next iteration is performed. 


3.3.5 Unconditional Jumps 

A GO TO statement causes control to jump to a statement with a 
specified label. The general form of the GO TO construct is: 

GO TO label; 

statement; 

statement; 


statement; 
label: statement; 

where "label" is a legal name, meaning that it must be a legal 
variable name. Any statement, except and END statement can have a 
label. Labels must be unique within a single procedure, although more 
than one GO TO statement can refer to a single label. 
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when a GO TO statement is executed, control is transferred 
the statement marked by the label specified in the GO TO. 


to 


3.4 Procedures 

Procedures in the DAT programming language are anf^^^ous to 
Qiihroutines in FORTRAN. A procedure has a name by which it can 
?ef«en«I? Ini allows for Arguments to be passed to and from it. 

The three statements which make procedures usable are 
PROCEDURE, CALL and RETURN. 


3 ^ 4,1 The PROCEDURE Statement and Arguments 

The PROCEDURE statement delimits and ?h 
f-ode as a functional unit. The end of the procedure is delimited with 
a corresponding END statement. The general form of a procedure is: 


PROCEDURE procedure-name I (arguments) ] ; 
statement; 
statement; 


END; 


statement; 


ThP. "orocedure-name" is the name given when CALLING a procedure. The 
"argumen?2" are data values that are supplied to the 

which Se procedure operates. Within a procedure definition, 
arquments are "dummy" variables having no value until the procedure i 
When the orocedure is called, the constants, variables or 
expressions given in the CALL statement are substituted for the dummy 
statements in^the PROCEDURE statement. The procedure then operates on 
?hr? el lvalues. The CALL statement is discussed more fully in 
^ The arquments in a PROCEDURE statement must be 

.Closed lA AaAentheses, Ind If there are more than one argument, they 
must be separated by commas. 


3.4.2 The RETURN Statement 


The RETURN statement is used to exit from a procedure. There 
may be more than one RETURN statement in a procedure. The general 
form of the RETURN statement is: 


RETURN [value]; 
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where the optional "value" may be a constantr variable, or expression. 
If the value is absent, the RETURN statement causes an immediate exit 
from the procedure in which it occurs, and a return to the place from 
which the procedure was called. The form of the RETURN statement with 
the value is used to transfer data from a procedure, as well as to 
exit a procedure; the "RETURN value" statement is used to make a 
procedure behave like a function. For example if the procedure FOO 
takes two arguments and returns a value, it could be invoked by a 
statement such as: 

NEWVAL = FOO(A,B); 

where NEWVAL is a variable to which the value returned is assigned, 
and A and B are the arguments supplied to FOO. 


3.4.3 The CALL Statement 

As noted in the previous section, a procedure that returns a 
value can be invoked as a function call, by setting a variable to the 
value of a procedure. This form of invocation cannot be used be used 
for procedures which do not return a value. To invoke such a 
procedure the CALL statement should be used. The form of the CALL 
statement is: 

CALL procedure-name [ (arguments) ] ; 

where "procedure-name" is the name given in the PROCEDURE statement 
defining the procedure, and the optional arguments are one or more 
constants, variables, or expressions, separated by commas. 


3.4.4 Commenting Procedures 

A procedure definition can contain statements that are not 
executed. These statements are called comments, and are usually used 
to describe the purpose and effects of the procedural statements. A 
comment is set off by the characters "/*" (slash, asterisk) at the 
beginning and "*/" at the end. These symbols and any text that occurs 
between them are ignored and never executed. Any characters may be 
included within the comments delimiters but the delimiters 
themselves. Comments may continue through any number of lines, and 
may be placed anywhere in the definition, except in the middle of a 
single word. 


3.5 Building a Procedure 

In the previous section, the definitive features of 
procedures, their uses, and the means by which they are invoked were 
discussed. In this section, the creation of procedures is discussed: 
how to enter them into the DAT subsystem. 
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3.5.1 Editing Basics 

The DAT subsystem includes a simple editor which is used to 
edit procedure definitions. The editor is a "line-oriented editor, 
similar to the EDIT program in use at LRC. 

To initiate editing of a procedure, the user types at command 

level : 

♦EDIT PRCX:eduRE procedure -name <G0> 

where "prooedure-name" can be the name o£ an existing pr^edure, which 
ia to be modified; or it can be the name of a new procedure, as yet 

undefined. 

If it is ordinary text that is being created or edited and not 
the definition of a procedure, the command would be: 


♦EDIT TEXT text-name <G0> 


where "text-name" is the variable name to use 
being edited. To have a PRCX:eD 0RE definition 
terminal, the command to use is: 


for the TEXT constant 
typed out at the 


♦TYPE definition OF procedure-name <G0> 


or 


♦TYPE DEF OF procedure-name <G0> 

On the other hand, to have a TEXT constant typed out at the terminal, 
the command to use is merely: 

♦TYPE text-name <G0> 

Once the EDIT command has been given to enter a new text, the 
EDITOR responds by displaying the following: 

0 <TOP OP TEXT> 

1 

2 <BOTTOM OF TEXT> 

If the text of a short procedure were to be input, using the editing 
commands listed below, the EDITOR might display. 

0 <TOP OF TEXT> 

1 PROCEDURE (A) ; 

2 TYPE A; 

3 END; 

4 

5 <BOTTOM OF TEXT> 
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Only the three statements on lines 1-3 are in the actual definition. 
The new line numbers are added automatically. The EDITOR updates and 
redisplays the text (renumbering the lines) after each editing 
command. For example, if the statement "A = A+1; <carriage-return>" 
were to be added after the PROCEDURE statement, the screen would 
display; 

0 <TOP OF TEXT> 

1 PROCEDURE (A) ; 

2 A = A+1; 

3 TYPE A; 

4 END ; 

5 

6 <BOTTOM OF TEXT> 

The EDITOR displays at most 25 lines of text on the screen; 
which lines are on the screen are under the user‘*s control via the 
screen commands discussed below. When lines are inserted into a full 
screen, the lower lines move off the bottom of the screen to 
accommodate the new lines above. The EDITOR does not permit editing 
ay off-screen lines. To edit a line, it must first be displayed on 
the screen. 


3.5.2 EDITOR Commands 

The EDITOR permits four kinds of actions; 

(1) FORWARD and BACK permit the user to control which portion of 

a long text appears on the screen; 

(2) INSERT, KILL, and CHANGE permit line-at-a-t ime changes to the 

text; 

(3) REPLACE permits replacements within a line; 

(4) EXIT and ABORT take the user from the EDITOR back to 

command- leve 1 . 

Any action which results in a change to the text causes the 
screen to be redisplayed. However, unless one of the screen commands 
has been given, redisplay begins with the same line at the top. 

Here are the detailed specifications of the commands (portions 
in brackets are optional) ; 


3. 5. 2.1 The FORWARD Command 

F[ORWARD] [line-count] <GO> 
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FORWARD moves text upward into the screen area. If the 
oDtional line-count is included, FORWARD will move ttat number of 
lines forward into the text. If the line-count is not included, 
forward moves the text so that the line following the as me 
currently displayed will be at the top of the new screen. 


3. 5. 2. 2 The BACK Conmand 


B[ACK] [line-count] <G0> 


BACK moves the text downward into 
line— count is not included, BACK moves the 
preceding the first line currently displayed 
the screen. 


the screen area. If the 
text so that the line 
will be at the bottom of 


3. 5. 2. 3 The INSERT Command 

I [NSERT after] line-number <G0> INSERT: 
one or more lines of text <G0> 

INSERT accepts lines of text (separated by carriage returns) 
and places them after line-number. The line-number argument is not 
optional. Note that two "<GO>"s are required — one after the 
co^aUr specification (following which the EDITOR types the word 
"INSERT:"), and one after the last line of input text. 


3. 5. 2. 4 The KILL Command 


K[ILL] first-line-number [[TO] last-line-number] <G0> 


KILL deletes line first-line-number, or 
first-line-number through last-line-number inclusive, 
the option specified. 


the lines 
depending upon 


3. 5. 2. 5 The CHANGE Command 

C[HANGE] first-line-number [[TO] last-line-number] <G0> INSERT: 
one or more lines of text <G0> 


CHANGE is exactly the equivalent to a KILL followed by an 
INSERT. It avoids an extra erase and redisplay of the screen, and 
implicitly inserts the specified text beginning at the same position 
as the text deleted. 


3. 5. 2. 6 The REPLACE Command 
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R[EPLACE] line-number <G0> REPLACE: text-to-match <G0> 

WITH: text-to-substitute <G0> 

Replace permits a single portion of a line to be replaced with 
another group of characters, without retyping the entire line. It 
finds the first occurrence in line line-number of the text-to-match, 
and replaces it with the specified text-to-substitute, then redisplays 
the screen. 

Note that three "<GO>"s are required: one after the command 
specification, following which the EDITOR types "REPLACE:"; one after 
the text-to-match, following which the EDITOR types "WITH:"; and one 
after the text-to-substitute. If the text-to-substitute is omitted, 
the text-to-match will merely be deleted. Carriage returns may be 
included in either text argument, and thus reduce or increase the 
number of lines. 


3. 5. 2. 7 The EXIT Command 
EX [IT] <G0> 

EXIT transfers control back to command-level, saving the newly 
edited version and replacing the old version with the new. 


3. 5. 2. 8 The ABORT Command 
ABORT <G0> 

ABORT transfers control back to command-level, leaving the 
original text unchanged. No changes that have been made in the EDITOR 
will be preserved. ABORT cannot be abbreviated in order to prevent 
its accidental execution. 
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4. The STAT Subsystem 


The purpose of the STAT subsystem 
scientific investigator to perform 
statistical inferences from his data. 


is to provide a tool for the 
statistical tests and make 


deJelopmen?" Lst of the etatlstloal P“^^9es under 
consideration have taken more than about five man-years 


4.1 Requirements of the STAT Subsystem 


In order to be 
statistical package would 
requirements : 


considered for use as the STAT subsystem, a 
have to meet the following four basic 


(1) Includes a large range of standard statistical analyses, 
including analysis of variance. 


(2) Runs interactively. 

(3) Runs on a CDC Cyber-175 computer. 

(4) Provides for data transformation. 


4.1.1 Standard Statistical Analyses 


The STAT subsystem should be capable of performing 
different types of statistical analysis. Some examples of classes of 
standard statistical analyses are the following: 


( 1 ) 


( 2 ) 


Descriotive statistics. This includes measures of central 
value (mean, median, mode) , measures of ^ 

(variance, standard deviation, differences between 

percentiles, etc.), measures tables 

agreement with a Gaussian distribution, along with tables 

and plots of frequency distributions, histograms, etc. 

Analysis of Variance. This analysis is used to determine the 
relationship and measure the contributions of a number of 
independent variables with a dependent variable, where the 
independent variables are categorical. ^his^ analysis is 
most relevant for the PM Module. One of the most frequently 
used experimental paradigms in research on performance 
measures is to make sets of runs with one (univariate) or 
more (multivariate) conditions at several discrete levels. 
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The outcome of these runs is a set of performance measures , 
some of which will have a relationship to the varied 
conditions. Analysis of variance is the technique used to 
measure these relationships. 

(3) Correlation and Regression Analysis. This analysis is also 

used to determine the relationship and measure the 

contributions of a number of independent variables with a 
dependent variable, but here all the variables are 

continuous. Scatter diagrams are also useful in examining 
these relationships. 

(4) Tests of a Model. A variety of statistical tests may be used 

to determine whether a set of experimental results differs 
from the predictions of a model. These include t-tests, 
chi-squared tests, etc. 

(5) Discriminant Analysis. This analysis is used to decide 

whether groups are different based on the information 
available in some number of analysis variables. 

(6) Factor Analysis. This refers to a number of techniques for 

analyzing correlation coefficients. The goal is to discover 

a few basic patterns or components in the relationships 
found in the correlation matrix. 

(7) Other Analysis Techniques. These techniques include 

non-pa rame tr ic tests, et al. 


4.1.2 Interactive Operation 

In order to allow for the efficient interchange of information 
between the analyst and the computer, the STAT subsystem must be 
interactive. The system must allow the analyst to interrogate the 
data base repeatedly and rapidly examine different statistical 
measures of different performance measures. It is also very 
desireable to be able to switch to a batch mode of operation once an 
analysis procedure is selected for use with a large data base. 


4.1.3 CDC Cyber-175 

The entire PM Module must run on The CDC Cyber-175 computer at 
LRC. This machine runs the NOS operating system. Any statistical 
packages which do not run on this system at present would have to be 
converted. In general for a package of this size, the conversion cost 
would be prohibitive, if it were even possible. 
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4.1.4 Data Transfocnation 


This requirement is the loosest. With a powerful 
transformation capability, a statistical analysis ^ 

fulfill virtually all the requirements of the entire ^ Module. a 
more modest goal is to meet the requirements of the Baseline PM 

Module. 


The basic requirements of data transformation are: 


(1) Edit data. Experimental data often contains values which are 

anomalous or erroneous, and which must be edited. 

(2) Select data. It is important to be able to specify and 

select a subset of the data for analysis. 


(3) Sort and Merge data. Data from various sources often has to 
be combined for joint analysis. For example, several runs 
from one subject might be combined for a subject summary, or 
several runs from several subjects under one condition might 
be combined. 


(4) Create new variables from old. This transformation 
capability would allow performance measures to be created 
from raw experimental results such as time series. There is 
virtually no limit to the degree of flexibility that would 
be useful here. 


4.2 Candidate Statistical Analysis Packages 

In the course of conducting the survey of existing, 
commercially available statistical analysis packages, on the order of 
sixty different packages received at least cursory consideration. 
They ranged from small, limited purpose packages to large general 
purpose packages; from free, unsupported software to multi-thousand 
dollar systems with annual upgrades; from card oriented batch systems 
to hiqhly conversational systems with on-line instruction; and from 
non-integrated subroutine libraries to fully integrated operating 
systems. The primary sources of information regarding these packages 
were two papers: Shucany, Minton and Shannon (1972), and Anderson and 
Sims (1977), as well as numerous conversations with many users or 
statistical packages. 


After a preliminary investigation of the packages, 
consideration was reduced to the following three: 


the number under 


(1) P-STAT 78, produced by P-STAT, Inc. of Princeton, New Jersey. 

(2) SIPS, produced by the Department of Statistics of Oregon 

State University, Corvallis, Oregon. 
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(3) SCSSf produced by SPSS Inc. of ChlcagOf Illinois. 

The first of these three candidate packages to be examined in 
detail was SCSS. It was found to have many strong features including 
a highly conversational front end, reasonable data transformation 
capabilities, and a moderate range of available statistical analyses. 
It had two major shortcomings, however: (1) It would not be available 
for operation on the CDC Cyber computer for at least six months or 
perhaps a year, and (2) It did not include analysis of variance. On 
the basis of these two shortcomings, SCSS was not given further 
consideration. 

The two remaining candidate packages, P-STAT 78 and SIPS were 
examined next. Their strength and weaknesses are discussed in the 
next section. 


4.3 Two Prime Candidates: P-STAT 78 and SIPS 

These two statistical analysis systems are compared by first 
considering how well they meet the four criterion listed above. Other 
factors, such as documentation, vendor support, and cost are then 
considered. 


4.3.1 Standard Statistical Analyses 

Both SIPS and P-STAT 78 include a wide range of standard 
statistical analyses, although SIPS has a significantly broader range 
than P-STAT 78. 

(1) Descriptive Statistics. Both packages compute a wide variety 

of descriptive statistics, along with tables and plots. 
P-STAT 78 provides a more complete and flexible facility for 
formatting the output for reports. 

(2) Analysis of Variance. P-STAT 78 contains a multi-variate 

analysis of variance subsystem. However, it is rather 
inconvenient to use; the software is not nearly up to the 
standards of the rest of the P-STAT package. SIPS contains 
adequate univariate and multivariate analysis of variance 
subsystems. 

(3) Correlation and Regression Analysis. Both P-STAT 78 and SIPS 

include fairly comprehensive correlation and regression 
analyses. 

(4) Tests of a Model. Both p-STAT 78 and SIPS include a variety 

of statistical tests of this type. SIPS offers a 
significantly wider range of tests, including several 
parametric and non-parametr ic tests. 
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( 5 ) 


Discriainant Analysis. P-STAT 78 includes discriminant 
analysis. SIPS includes discriminant analysis a part of the 
MANOVA subsystem. 


( 6 ) 


Factor Analysis. Both P-STAT 78 and SIPS include commands 
for performing factor (principal components) analysis of a 
covariance matrix, and for performing rotations based on 
this analysis. 


(7) Other Analysis Techniques. S 
non-parametr ic tests such 
Kruskal-Wallis, sign test, etc. 


includes a variety of 
Wilcoxon, Mann-Whitney, 


4,3.2 Interactive Operation 

Both the P-STAT 78 and SIPS systems operate interactively; 
both allow the user to rapidly explore a set of statistical analyses, 
to drop one and pursue another as the results are obtained. P STAT 7 
provides a greater level of diagnostic error messages and a more 
elegant scheme of error recovery. The glaring exception to this 
feature occurs in the P-STAT 78 Analysis of Variance routines where a 
simple user error can crash the entire P-STAT system. 

Both the P-STAT 78 and SIPS systems provide convenient 
mechanisms for operating in a batch mode, as might be desired 
analyzing a large data base. P-STAT provides 

for setting up the batch run; the user has the ability to edit the 
typescript generated in a smaller interactive run. 


4.3.3 CDC Cyber-175 

Both the P-STAT 78 and SIPS systems are written primarily in 
FORTRAN, and will operate on the CDC Cyber-175 computer at LRC. 
P-STAT is written in machine independent form, and has been made to 
run on a CDC Cyber system by means of a preprocessor conversion 
program. SIPS has been developed exclusively on CDC computers and has 
not yet been converted for use on other machines. It _is highly 
optiLzed for use on the CDC Cyber computers under the NOS operating 
system, and should run significantly faster than P-STAT on the 

Cyber-175 at LRC. 


4.3.4 Data Transformation 

Both P-STAT 78 and SIPS provide mechanisms for fairly 
extensive data transformation. Although neither system would fulfill 
all the requirements for data transformation for the entire M Module, 
either one would meet the requirements of the baseline PM Module. 
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Both systems allow for editing data, selecting data, and 
sorting and merging data. In addition, both systems allow new 

variable to be created from old ones. The ability to create new 
variables from arbitrary sets of old variables is limited. 


4.3.5 Other Considerations 

The P-STAT and SIPS systems have similar origins; both began 
as statistical analysis systems serving a university computing center. 
P-STAT was originally developed at Princeton in 1962, and SIPS at 
Oregon State University in the early 1970‘*s. The aims and directions 
taken by the two systems following their initial development, however, 
have been somewhat different. 

The emphasis in P-STAT has been on making the system appealing 
to as broad a community as possible. The following attributes of the 
P-STAT system result, at least in part, from this emphasis: 

(1) The documentation for P-STAT is very complete and up-to-date. 

Each feature of the system is explained in some detail. 

(2) The system works on a wide variety of computer systems 

(including IBM, UNIVAC, DEC, CDC, Honeywell, Burroughs, 
PRIME, INTERDATA, Hewlett-Packard, and Harris), although it 
is probably not highly optimized on many of them. 

(3) A private corporation, P-STAT, Inc., has been established to 

sell and support the P-STAT system. Consequently, the 
problems encountered by users are handled effectively. 

(4) The interactive front-end (the user interface) of P-STAT is 

quite sophisticated. When P-STAT is used interactively, an 
editor file is kept which contains both the commands and 
data records which have been typed in. This file can be 
accessed at any time to fix errors, or to add, delete, or 
replace commands or data. The corrected commands can then 
be re-executed. The editor file can be saved for use in 
another interactive session of for submission as a batch 
job. 

(5) There is great flexibility in providing well formatted tables 

and graphs, which are suitable for inclusion "as-is" in 
reports. 

(6) Although the range of available statistical analyses is 

large, there has not been an effort to maintain an 
exhaustive set of analyses. 

(7) P-STAT contains commands which read and write BMDP and SPSS 

system files. Thus, the P-STAT user has access to two of 
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the most powerful and widely 
statistical analysis systems. 


available batch type 


(8) In short, the emphasis is on ease of use, rather than 
statistical completeness. 


on 


in contrast, the emphasis In SIPS has been on making the 
svstein as useful and as powerful as possible for people 9 

iCtruniverlity. The following attributes of the SIPS system result, 
at least in part, from this emphasis: 

(1) Documentation for SIPS is adequate but not 

users would probably require at least occasional telephone 

consultation with the SIPS staff. 

The system is highly optimized for the CDC Cyber computer 
^ ^ s?sSm it aW not currently work on other computer 
systems, although such conversions may be done in the 
future. 

(3) The Department of Statistics of Oregon State University 

maintains the SIPS system. User problems must be ^andled^by 
this group only on a part-time basis. During the SIPS trial 
period at LRC, user problems were handled effectively. 

(4) The interactive front-end (the user-interface) ® 

straightforward command processor with good error checking 
etc. As a result, the system is very efficient use. 

(5) The output is generally presented in a clear and readable 

format, although there is not much flexibility in formatting 
tables and graphs. 

(61 There is a large emphasis on providing as complete a set of 
statistical analyses as possible. New functions are 
continuously being added. However, there are no convenient 
links available to BMDP or SPSS. 

(7) In short, the emphasis is on statistical completeness, rather 
than on universal useability. 

The cost of the P-STAT system to LRC would be $5000 for the 
first year and $2000 for subsequent years; the cost of the SIPS 
system^ would be $5000 for the first year and nothing for subsequent 

years. 


4.4 A Recommendation 

Either the P-STAT 78 or the SIPS system would meet the minimum 
requirements of the STAT subsystem. However, it is felt that the 
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superior statistical capabilities of the SIPS system, especially in 
the area of analysis of variance, far outweighs the I/O advantages of 
'P—STJi’I for the intended usage in the PM Module. The expected superior 
efficiency of the SIPS system is another strong factor. For these 
reasons, it is recommended that LRC purchase the SIPS system for use 
as the STAT subsystem in the PM Module. 
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5. A Consistent File System for the PM Module 

The Consistent File System links the DAT and STAT subsystems. 
This file system provides a means for passing data from the output of 
the experiments through the DAT and STAT subsystems. The primary task 
of the Consistent Pile System is to maintain the correspondence 
between the value of the data (e.g. the value of a particular 
performance measure) and the identities of the data (e.g, the name of 
the performance measure) . 

The design of the Consistent File System for the PM Module has 
two distinct parts: the design of internal file structures and the 
design of external file structures. Internal file structures means 
the choice of header content, record size, etc.: how does one identify 
or find a particular datum within a file. External file structures 
means the choice of file naming and accessing conventions: how does 
one identify and retrieve a particular file for analysis. 


5.1 Internal File Structures 


5.1.1 Design Objectives 

The internal file structures of the Consistent File System are 
designed meet the following objectives: 

(1) Adaptable, so that a wide variety of experiments and analyses 

may be accommodated. The system must handle experiments and 
performance measures which have not yet been conceived. 

(2) Self-docuHienting, so that the files may be shared by^ users 

with little contact with each other. To achieve this, the 
system should store both the value and the identity (name, 
units, type, etc.) of each datum. 

(3) Expandable, so that a subset of the users can make an 

addition to the file system with no impact on the other 
users, and without making existing files unreadable. 

(4) Upward coupatible, so that features which are not 

incorporated during the initial development can be added 
later . 

(5) Backward coa^atible, so that existing file handling software 

(e.g. SIFT) can be used. 

(6) Convenient, so that reading and writing the files requires 

just a few lines of code. 
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(7) Efficientf so that the computing resources^ such as CPU time/ 
disk space/ memory/ etc./ are not unduly burdened. 


5.1.2 SIFT Piles 

Over the past several years a file structure/ or format/ known 
as SIFT was developed at LRC. The goals of the SIFT file system were 
similar to the goals of the PM Consistent File System. It is not 
surprising then/ that many/ although not all/ of the design objectives 
of the Consistent File System are met by SIFT. In this section/ the 
SIFT file system is described/ and its strengths and weaknesses 
indicated. 

Figure 5.1 illustrates the format of a SIFT file. The file is 
divided into blocks/ and each block is divided into header and data 
sections. The header section describes the layout or format of a 
record in the data section. The data section contains a set of data 
records in this format. 

The header section consists of four records which specify the 
number of variables in each data record/ and the name (e.g. input or 
error)/ units (e.g. grams, meters), a power of ten (e.g. 10**3), and a 
reference value for each variable. 

The data section consists of any number of data records. Each 
record contains one value for each variable. The data section is 

terminated by a special END record. 

The SIFT file system has the following strengths: 

(1) Self-documenting. Each datum is identified by name and 

units. 

(2) Expandable. Subsets of users may add variables to a 

particular file with little or no impact on other users. 
The limit of expandability is determined by the size of the 
read/write buffers. 

(3) Convenience. It is extremely convenient for users to access 

SIFT files. Simple FORTRAN read and write statements are 
all that is required. 

( 4 ) Efficiency. Accessing SIFT files consumes little CPU time or 

program memory, since read and write requests are simple. 

On the other hand, the SIFT file system appears to have the 
following weaknesses: 

(1) Adaptability. A serious limitation on the adaptability of 
SIFT files is that they cannot handle arrays conveniently. 
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Header 

Section 

^NAME', N, FNAME(N) 
^UNIT', N, FUNIT(N) 
'ID', N, FDECML(N) 

'ZERO', N, FREF(N) 

Block 1 

Data 

Section 

'DATA', N, DATA(N) 
'DATA', N, DATA(N) 

• • • 

• • • 



'DATA', N, DATA(N) 
'END', N, DATA(N) 


Header 

Section 

'NAME', N, FNAME(N) 
'UNIT', N, FUNIT(N) 
'ID', N, FDECML(N) 

'ZERO', N, FREF (N) 

Block 2 

Data 

Section 

'DATA', N, DATA(N) 
'DATA', N, DATA(N) 

• • • 

• • • 



'DATA', N, DATA(N) 
'END', N, DATA(N) 


Figure 5.1 Format of a SIFT File 
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(There is no 5HDIMEN key, and N for data records would have 
to differ from N for all other records.) In addition, only 
real (floating point) numbers are currently handled, 
although some other (single-word) types could probably be 
added easily. 

(2) Efficiency. SIFT files may make inefficient use of disk space 

since the record size is variable. The minimum size ^ 
physical disk record is 64 words, so that if the record size 
were 16 (which is a reasonable size for scalar data) then 
3/4 of the disk space used would be wasted. In addition, 
the user program must allocate a core buffer which is large 
enough to handle any record which it might encounter. Also, 
since these files are accessed sequentially, if some 
application requires random read access it will be very 
slow; random write access may be impossible. Finally, an 
application requiring very long records may be limited by 
the maximum physical record size allowed by the NOS 
operating system. 

(3) Upward cCTopatibility. Some features, such as vectors, could 

probably be implemented later in an upward compatible 
fashion. Other features, however, such as random access or 
uniform record length, are probably excluded. 


5.1.3 SIFT Extended (SX) Files 

In order to fulfill more of the design objectives of the PM 
Consistent File System, we have developed a file structure which is a 
natural extension of SIFT. This structure, which we call SIFT 
Extended or SX, combines the basic concepts of SIFT with a set of 
additional features. 

As currently developed, the SX file structure may be thought 
of as being like the SIFT structure with a set of extensions. Many of 
these extensions are optional; they may be implemented or not, 
depending on the priorities of the LRC staff. 

The following list of the SX extensions is divided into three 
groups. It is recommended that the first group be given a high 
priority for implementation. Many of the objectives of the m 
Consistent File system would be met by implementing this group. It is 
recommended that the second group of extensions be given ^ a medium 
priority. The extensions in this group are designed to improve the 
efficiency and increase the flexibility of the SX file system. It is 
recommended that the third group be given a low priority, and that 
their implementation be deferred until some experience with the SX 
file system has been obtained. 
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The following is a list of the high priority SX extensions to 
the SIFT file system. These extensions could be implemente qu e 
simply, using FORTRAN READ and WRITE statements; no extra software 

would need to be written. 

(1) Arrays. The SX files will support array data conveniently. 

' ^ There would be a header record which specified the size 

(i.e. dimension) of each variable in a data record. 

(2) variable types. The SX files will support varied types 

conveniently. The basic types will be RBTU^ , INTEGER , ad 
CHARACTER. The SX routines will not use this information, 
however; it will merely be storable and retrievable. 
Therefore, users will be able to add their own types, such 
as DATE, etc. 

C3» Expanded Header. In the SIFT file system, the header 
' ’ ^contains a description of the format of the data records. 
In the SX file system, the header would be expanded so that 
it contained its own "data" record, called a header 
It would then contain a description of that record, along 
with a description of the data records. For example, the 
header record might serve to identify a run of experiment 
(e.g. the date, subject, experimental conditions, etc.), 
while the data records would contain the results of the 
experiment (e^g. time series of input, error, etc.). 

The following is a list of the medium priority SX extensions 
to the SIFT file system. Unlike the first set of extensions, t 
extensions would be implemented in a set of SX 

Therefore access to the SX files would not be via FORTRAN READ and 
WRITE statements, but rather via calls to these routines. 

(4) Buffered File Records. To avoid the inefficiencies of very 

small disk file records, the _problem of a maximum disk 
record size, and the requirement for a core buffer large 
enough for the largest record, SX file access will be 
buffered. Actual disk records will be a uniform size 
(probably 128 words) , and are distinct from SX records which 
can be any size. Whenever an SX record is accessed, the 
required disk records will be automatically 

out as required* A secondary advantage of buffered file 
records is that there is a considerable efficiency gain when 
the file records are small; many file records are read into 
memory with a single disk access* 

(5) Random Access. The SX files will be accessible via random 

access. Readers will be able to access any existing record. 
In addition, random access will permit header information, 
such as the location of the last record or the time the file 
was last written into, to be updated as needed. The current 
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NOS FORTRAN supports random access files (OPENMS, REAIMS and 
WRITMS) . However, it imposes the requirement that an array 
be kept in the users memory space (FL) of length equal to 
one plus the maximum number of disk records in the file. 
This could be a significant problem if several large SX 
files must be open concurrently. It is expected that a 
FORTRAN 77 conforming compiler will become available within 
a year or so; such a compiler should support random access 
without this overhead. 

(6) Read/Write Conventions. Data may be written into an SX file 

as an entire record or as a set of fields within a record. 
Data is read from an SX file only as a set of fields within 
a record. This restriction on readers is intended to 
guarantee the expandability of the SX file structure. For 
example, if a reader assumed that the position of the fields 
within a record was not going to change, he might read whole 
records and extract the fields himself. The resulting code, 
however, would be vulnerable to a change in the structure of 
a record, such as the addition of a new field. This 
extension would, however, impose an inefficiency, hopefully 
small, on readers. 

The following three additional extensions are given low 
priority; their implementation should probably be deferred until some 
experience with the SX file system has been obtained. However, care 
should be taken in the design not to preclude incorporating these 
extensions at a later time. 

(7) SX Supplied Fields. The SX file system might automatically 

supply one or two fields a each record is created. Two 
possibilities are Record-Number and Entry-Time. 

(8) Access Control. The SX file system should probably contain 

some kind of access control. This control would have to 
work within the constraints imposed by the NOS operating 
system. A minimum form of access control would require 
users to specify a mode of access (e.g. read, append, or 
write) when they opened an SX file. The SX routines would 
then enforce access mode restrictions. 

(8) SX Directory. For many applications where SX files contained 
multiple Header/Data blocks, it would be useful to have a 
directory of the blocks. This directory would be written 
before the first block. 


5.1.4 SX File Structure 

Figure 5.2 illustrates the format of one block of an SX file. 
As in SIFT files, the header section describes the layout or format of 
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LPH, NREC, DATBCR, TIMECR, USERCR | 

Header 

Header Record Descriptor; 

NFLDH, FNAMEH (NFLDH) , FSI ZEH (NFLDH) , 
FONITH (NFLDH) , FTYPEH (NFLDH) 

Section 

Header Record; I 

Field 1, Field 2, ... , Field NFLDH | 


Data Record Descriptor; 

NFLD, FNAME(NFLD), FSIZE(NFLD), 
FUNIT (NFLD) , FTYPE (NFLD) 


Data Record 1; 

Field 1, Field 2, ... , Field NFLD 

Data 

Section 

Data Record 2; I 

Field Ir Field 2, ... , Field NFLD | 

• • • 

• • • 

• • • 

Data Record NREC; I 

Field 1, Field 2, ... , Field NFLD 1 


Figure 5.2 Format of One Block of an SX File 
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a record in the data section, and the data section contains a set of 
records in this format. The major difference between the SX and SIFT 
files blocks is that the SX header contains additional information. 

The Header section contains four records: a pre-header, a 
header record descriptor, a header record, and a data record 
descriptor. The Data section contains data records. All of these 
records in both the Header and data sections are divided into fields. 
The fields of the pre-header, header record descriptor, and data 
record descriptor are fixed. The fields of the header record and data 
records are determined by the users. These fields are all described 
below and are summarized in Table 5.1. 

The pre-header contains the length of the pre— header (LPH) , 
and the number of records in the data section (NREC) . It might also 
contain the date and time the block was created (DATBCR, TIMECR) as 
well as the user who created the block (USERCR) . The header record 
descriptor contains the number of fields in the header record (HFLDH) , 
and the name, size units and type of each field (FNAMKH (NFLDH) , 
PSIZEH(NFLDH) , FONITH (NFLDH) , FTYPEH (NFLDH) ) . The header record 
contains NFLDH fields as described by the header record descriptor. 
The data record descriptor is analogous to the header record 
descriptor and contains the number of fields in each data record 
(NFLD) , and the name, size, units and type of each field (FNAME(NFLD) , 
FSIZE(NFLD) , FDNIT(NFLD), FTIPE (NFLD ) ) . The data records contain NFLD 
fields as described by the data record descriptor. 

Figure 5.3 illustrates the overall format of an SX file with 
multiple blocks. The first block, called the SX Header contains the 
location of the SX Directory (LOCDIR) , the location of the current EOF 
(LOCBOF) , the current number of blocks (HBLK) , and the maximum number 
of allowable blocks (MAXBLK) . The SX Directory contains the location 
of the start of each block in the SX file. 


5.1.5 SX File Access 

In this section, the means of accessing SX files is described. 
As noted in Section 5.1.3, if only the first three high priority SX 
extensions to the SIFT file system are implemented, then the SX files 
could be accessed using just FORTRAN READ and WRITE statements; no 
extra software would need to be written. If however, additional 
extensions are desired, then the following set of SX utility 
subroutines would provide users with a convenient mechanism for 
accessing SX files. The following list of routines would allow users 
to open and close SX files, to read and write SX record descriptors, 
and to read and write SX records. 

(1) Opening SX Files. 
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SX Header 


LOCDIR, LOCEOP, NBLK, MAXBLK 


SX Directory 


Location of Block 1 
Location of Block 2 


Location of Block NBLK 


Location of Block MAXBLK 


i 

Header Section I 

Block 1 

1 

T 

Data Section I 

Header Section I 

Block 2 

1 

Data Section I 

• • 

• 

• 

• • 

• • 

• 

1 j 

Header Section I 

Jk5±OCK 

1 1 

Data Section 1 


Figure 5.3 


Overall Format of an SX File with Multiple Blocks 



SXNEN: Opens a new SX file 
SXOLD: Opens an existing SX file 

These two routines require that the files to be opened be 
specified by n^uBe at run-time. The current NOS FORTRAN 
compiler does not support this capability. A set of 
assembly language (COMPASS) routines with this capability, 
however, has been developed at LRC. In addition, a 
FORTRAN 77 conforming compiler, which should become 
available within about a year, would also support this 
capability. 

(2) Writing SX Record Descriptors. 

SXDEFHt Writes a header record descriptor 
SXDEP: Writes a data record descriptor 

(3) Reading SX Record Descriptors. 

SXRECH: Returns information about a set of fields of a 

header record, specified by field numbers. 

SXFLDH: Returns information about a set of fields of a 

header record, specified by field names. 

SXRBC: Returns information about a set of fields of a data 
record, specified by field numbers. 

SXFLD: Returns information about a set of fields of a data 
record, specified by field names. 

(4) Writing SX Records. 

SXWRRH: Writes an entire header record. 

SXWRFH: Writes a set of fields in a header record, specified 
by field numbers. 

SXWRR: Writes an entire data record. 

SXWRF: Writes a set of fields in a data record, specified by 
field numbers. 

(5) Reading SX Records. 

SXRDFH: Reads a set of fields from the header record, 
specified by field numbers. 

SXRDF: Reads a set of fields from a data record, specified 
by field numbers. 

Note that header and data record descriptors are written as 
entire records, whereas they may be read by field names or field 
numbers. This is because while writers would presumably know what 
fields are to be written, readers might not know any of the field 
names or might desire to search for a specific field name. Also, 
writers can access header and data records by either entire records or 
by field numbers, whereas readers must access these records by field 
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numbers (having obtalnea these field numbers by reading header or data 
record descriptors) . 


5.2 External Pile Structures 


5.2.1 Design Objectives 

The external file structures of the Consistent File System 
cimoiifv the task of identifying and retrieving a particular file for 
analysis ^ The system is in fact a file-naming convention, supported 
S appropriate so/tware, with multi-field file 
would serve to identify the following attributes of the file. 


(1) The user who "owns" the file: his group and his name. 

(2) The experiment to which the file pertains: the name of the 

experiment, the pilot and run number. 


(3) The type of file: time-series data, processed data, etc 


(4) The version number of the file. 

These file naming conventions would probably also be very 
useful f« Kepinrtrack of programs pertaining to the PM Module, such 
as FORTRAN source files, documentation files, etc. 


5.2.2 Fields of Pile Names 

Pile names are divided into four fields. User, Experiment, 

Type, and Version, corresponding to ® 

file. The fields could be separated by a character such as / . 

User/Exper iment/Type/Ver s ion 

TO accomodate files of other types, i.e. not experimental^ da t^ the 
Experiment field could be given the more general name of PileName. 


User /Pi leName/Type/Ver s ion 


These fields could, in turn, be further divided into 
subfields, to more specifically identify the file, /he sub-fields 
could be separated by a character such as .. Thus a file name witn 
four fields could have the following sub-fields. 


User ==> Group. Use rName 

FileName ==> Experiment. Pi lot. Run 
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Type 

Version 


==> Major Type. Minor Type 

==> MajorVersion.MinorVersion 


5.2.3 Implenenting External File Structures 

Implementing these external file structures, i.e. long, 
multi-field file names, directly under the NOS operating system would 
be extremely difficult, if not impossible. The NOS operating system 
is limited to file names of seven letters or less, with each user 
having his own set of file names. A feasable approach, however, would 
be to write a program which somehow maintained a correspondence 
between long user-oriented file names, and short machine -oriented file 
names. At the present time, two programs are in existence at LRC, 
which approach this capability: The Pile Information and Cataloging 
System (PICS), and The Permanent Pile Cataloging System (CCATSYS) . 
The applicability of these two programs to the needs of the PM 
external file structures is discussed below. 


5. 2. 3.1 File Information and Cataloging System (FICS) 

PICS provides a method for identifying, cataloging and 
displaying contents of the permanent file system. The system is built 
as an inverted tree structure, composed of information nodes. The top 
level node is called the root node. Below the root node are other 
nodes, connected to each other by branches. Some of the nodes, called 
data base nodes, are associated with file entries which describe 
actual NOS files. Each data base node may have file entries for a 
number of NOS files. 

One way of using PICS would be to associate the PICS node 
levels with the User field and its sub-fields Group, UserName, or 
perhaps some others. The Type field would be handled by the "Type 
Description" portion of the file entry in the data base. The 
remaining fields, PileName and Version, would have to be handled by 
the 80 character Pile Description provided by PICS. 

Another way of creating the above external file structures 
using FICS would be to associate each node level with a field (or 
subfield) , and each node at that level with an instance of that field 
(or subfield) . The file entry could then be used to complete the file 
name and to store other information about the file. A disadvantage of 
this approach is that only a priviledged user, called the Tree Boss, 
is allowed to create new nodes. Therefore, only a Tree Boss can name 
files. 

There are three disadvantages of FICS. First, the system 
appears to be somewhat cumbersome. Although practice users might 
become proficient, it seems that for this application FICS requires an 
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unduly large ^hiir'Sor tl«lbir°lncon»e"i®^'^ 

remember to perform toe - associating seven letter file names 

although FICS provides a means of /gtiH required to invent 

with longer, more for all their files. A possible 

exSrsior'to'^Pl'rs ";iuld" be for the 

names which the user might never even see directly. 

5. 2. 3. 2 Permanent File Cataloging System (CCATSYS) 

CCATSYS provides another system for CCATSYS^il 

and discing intents o£ the '1 ringTi ’S.tTLsl 

somewhat smaller than FICS, and suitable for each user to maintain 

node of FICS. Consequently ,. it is suitaoieror^e^^^^ ^ manner 

his CCATSYS oatal^, ^^viating^ the^^ «>®/^i%HP® 

Similar to PiCS, tne ,^0 fields FileName and Version 

'ruil“L°‘h™'br'th‘r irraJalt.^ m. Description provided by 
CCATSYS . 

CCATSYS has one major |dvanta|e over CCATSYS 

some of the disadvantages * . easier than FICS. On the 

appears to be |jCS ^ the 'procedure for adding a new file to 

other hand, as with FI ' « 5 hni reauired to generate unique 
CCATSYS is manual, and users are s disadvantage of CCATSYS is that 

seven letter file names. The maDor ® versus the 80 

only 37 character File FICS In addition, whereas a FICS 

character descriptions J g users, individual CCATSYS files 

;^r?:gu'l?ed“£orra=h’u^er!* r^th^ °o^hef£l d " CCATSYS users could 
eHurteid each others CCATSYS flies and printouts. 

overall, It .«PP®®«,.‘*'®‘ CCMSY|^jould ^ Jhe^catal^lng 
system of choice for J^plemen i ^e of CCATSYS far outweighs the 

SrsadlanJagT ot ‘nIt'LMnrabirtrshare the cataloging system across 
users. 

5 3 Examples of Internal and External Pile Structures 

T-iu-”?;”,:? 

S“,s“v.;s' 

(1977) for the details of this study. 
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5.3.1 Single-Run Data Piles 

Each run of the experiment would produce a Single-Run data 
file with a name such as; 

LRC-SIM . MCKISSICK/G-SEAT . DEF . 4/SX . DATA/1 .0 

which would correspond to subject DEF and run 4. If the run had to be 
redone for some reason, the new file might be version 1.1. The header 
record of this file would include the following fields which identify 
the file, and possibly some others which more specifically identify 
the experimental conditions: 


NFLOH = 12 

FNAMEH FSIZBH 

Group 1 

UserName 1 

Experiment 1 

Pilot 1 

Run 1 

MajorType 1 

MinorType 1 

MajorVsn 1 

MinorVsn 1 

Condition 1 

Sample Rate 1 

Date 1 


The data records 
each system state: 


FONITH 


SAMP/SEC 


FTYPEH 

VALUE 

CHAR 

LRC-SIM 

CHAR 

MCKISSICK 

CHAR 

G-SEAT 

CHAR 

DEF 

CHAR 

4 

CHAR 

SX 

CHAR 

DATA 

INTEGER 

1 

INTEGER 

0 

CHAR 

SEAT-ON 

REAL 

16.0 

DATE 

ll-FEB-79 

iist of eleven fields. 


NPLD = 11 




FMAME 

FSIZE 

FONIT 

ETYPE 

TKE 

1 

DEGREES 

REAL 

TKL 

1 

DEGREES 

REAL 

TKC 

• 

1 

« 

DEGREES 

• 

REAL 

• 

• 

• 

T 

« 

• 

1 

• 

• 

SAMPLES 

• 

• 

INTEGER 
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5.3.2 Single-Run Measureaent Piles 

One analysis scheme would be to analyze each such 
data file to produce a corresponding Single-Run measurement 


Single-Run 

file: 


LRC-SIM . MCKISSICK/G-SEAT . DEF . 4/SX . MEAS/1 . 0 


The header of this file would include the same 
data file except that the file type wou 
SX.DATA. 


fields as the preceding 
Id be SX.MEAS instead of 


NFLDH = 12 


FNAMEH 

Group 
UserName 
Exper iment 
Pilot 
Run 

Major Type 

MinorType 

MajorVsn 

MinorVsn 

Condition 

Sample Rate 

Date 


FSIZEH 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


each of the 90 performance measures: 


FUNITH 

FTYPEH 

VALUE 


CHAR 

LRC-SIM 

M 

CHAR 

MCKISSICK 

_ 

CHAR 

G-SEAT 


CHAR 

DEF 


CHAR 

4 

— 

CHAR 

SX 

_ 

CHAR 

MEAS 


INTEGER 

1 

— 

INTEGER 

0 


CHAR 

SEAT- ON 

SAMP/SEC 

REAL 

16.0 

— 

DATE 

ll-FEB-79 

a single 

data record 

with one fl 


for 


NPLD = 90 




FRAME 

FSIZE 

FONIT 

FTYPE 

MTKE 

1 

DEGREES 

REAL 

MTKL 

1 

DEGREES 

REAL 

MTKC 

• 

1 

• 

DEGREES 

• 

REAL 

• 

• 

• 

TS+ 

• 

• 

1 

• 

• 

SECONDS 

■ 

• 

REAL 

TS- 

1 

SECONDS 

REAL 



5.3.3 Pilot-SuBsary Neasureaent Files 

Later, one could combine these single-run measurement files 
across runs to produce pilot-summary measurement files: 

LRC-SIM.MCKISSICK/G-SEAT .DEP .ALL/SX.MEAS/1. 0 

where ALL indicates all runs. The header record of this file would 
consist of the following fields: 


NFLDH = 10 



It 

11 

11 

11 

II 

II 

11 

11 

II 

11 

II 

II 

11 

II 

II 

11 

II 

II 

II 

II 

FNAMEH 

FSIZEH 

FDNITH 

FTYPEH 

VALUE 

Group 

1 

— 

CHAR 

LRC-SIM 

User Name 

1 

- 

CHAR 

MCKISSICK 

Experiment 

1 

- 

CHAR 

G-SEAT 

Pilot 

1 

- 

CHAR 

DEP 

Run 

1 

- 

CHAR 

ALL 

Major Type 

1 

- 

CHAR 

SX 

MinorType 

1 

— 

CHAR 

DATA 

MajorVsn 

1 

— 

INTEGER 

1 

MinorVsn 

1 

- 

INTEGER 

1 

Sample Rate 

1 

SAMP/SEC 

REAL 

16.0 


II 

II 

11 

11 

II 

11 

11 

It 

11 

11 

It 

11 

II 

II 

II 

II 

II 

II 

11 

II 

11 

11 

11 

11 




The data records of this file would consist of three fields 
identifying the run, plus fields for the 90 performance measures: 


NFLD = 93 


FNAME 

FSIZE 

FONIT 

FTYP] 

Run 

1 

— 

CHAR 

Condition 

1 

- 

CHAR 

Date 

1 

- 

DATE 

MTKE 

1 

DEGREES 

REAL 

MTKL 

1 

DEGREES 

REAL 

MTKC 

• 

1 

• 

DEGREES 

• 

REAL 

• 

• 

• 

TS+ 

• 

• 

1 

• 

• 

SECONDS 

• 

• 

REAL 

TS- 

1 

SECONDS 

REAL 
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5 3.4 overall-SiMMiry Measureaent Piles 

still later, one could coaO,ine the measurement 

files to produce an Overall-Summary measurement fxle. 

lrc-sim.mckissick/g-seat .all .ALL/SX.MEAS/1. 0 

where ALL.hLL represents all ®fia™ns. The header record 

of this file would include the following field . 







nfldh = 10 





FNAHEH 

FSIZEH 

FOMITH 

FTYPEH 

VALUE 




CHAR 

lrc-sim 

Group 

1 


CHAR 

MCKISSICK 

UserName 

1 


CHAR 

G-SEAT 

Exper iment 

1 


CHAR 

ALL 

Pilot 

1 

"1 


CHAR 

ALL 

Run 

1 


CHAR 

SX 

Major Type 

1 


ch;^ 

DATA 

MinorType 

1 


INTEGER 

1 

MajorVSh 

1 


INTEGER 

1 

MifibrVsri 

SamnleRate 

1 

1 

SAMP/SEC 

REAL 

II 

II 

11 

II 

II 

o II 

• 1 

VO 1 
fH 1 
1 
1 
1 
1 


fhis file would consist of four fields identifying 
tSe pflSt’^lnrtLlunfpl^siields for the 90 performance measures: 


NFLD = 94 




FNAME 

FSIZE 

FONiT 

fptpe 

Pilot 

Run 

Condition 

Date 

MTKE 

MTKL 

MTKC 

• 

1 

1 

1 

1 

1 

1 

1 

• 

degrees 

degrees 

degrees 

• 

CHAR 

CHAR 

CHAR 

DATE 

REAL 

REAL 

REAL 

• 

• 

• 

TS+ 

TS- 

• 

• 

1 

1 

• 

seconds 

seconds 

REAL 

REAL 
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Finally, this Overall-Summary measurement file would be in a 
form suitable for statistical analysis by the STAT subsystem. In 
fact, one could obtain preliminary statistical results from this 
experiment by analyzing Pilot-Summary files or even Single-Run files 
with the STAT subsystem. 
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